{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 使用多个界定符分割字符串\n",
    "- re.split()\n",
    "- str.split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['asdf', 'fjdk', 'afed', 'fjek', 'asdf', 'foo'],\n",
       " ['asdf', ' ', 'fjdk', ';', 'afed', ',', 'fjek', ',', 'asdf', ',', 'foo'],\n",
       " ['asdf', 'fjdk', 'afed', 'fjek', 'asdf', 'foo'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "line = 'asdf fjdk; afed, fjek,asdf, foo'\n",
    "\n",
    "列表 = re.split(r'[;,\\s]\\s*', line)\n",
    "括号 = re.split(r'(;|,|\\s)\\s*', line)\n",
    "去逗号 = re.split(r'(?:,|;|\\s)\\s*', line)\n",
    "列表, 括号, 去逗号"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 字符串开头或结尾匹配\n",
    "- startwith()\n",
    "- endwith()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 用Shell通配符匹配字符串\n",
    "- fnmath, fnmathcase\n",
    "- fnmatch() 函数使用底层操作系统的大小写敏感规则(不同的系统是不一样的)来匹配模式\n",
    "- fnmatch() 函数匹配能力介于简单的字符串方法和强大的正则表达式之间。 如果在数据处理操作中只需要简单的通配符就能完成的时候，这通常是一个比较合理的方案"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, False)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from fnmatch import fnmatch, fnmatchcase\n",
    "\n",
    "fnmatch('foo.txt', '*.txt'), fnmatchcase('foo.txt', '*.TXT')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 字符串匹配和搜索\n",
    "- str.match()\n",
    "- str.findall()\n",
    "- re.compile()\n",
    "- re.findall()\n",
    "- re.finditer() 迭代方式返回匹配"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 字符串搜索和替换\n",
    "- str.replace()\n",
    "- re.sub()\n",
    "- re.sub() # Return result and n 改变的次数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Today is 2012-11-27. PyCon starts 2013-3-13.'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = 'Today is 11/27/2012. PyCon starts 3/13/2013.'\n",
    "import re\n",
    "datepat = re.compile(r'(\\d+)/(\\d+)/(\\d+)')\n",
    "datepat.sub(r'\\3-\\1-\\2', text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 字符串忽略大小写的搜索替换\n",
    "- re.IGNORECASE 标志"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.7 最短匹配模式\n",
    "- 默认是贪婪匹配\n",
    "- 加上?修饰符非贪婪"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.8 多行匹配模式\n",
    "- 无法匹配多行\n",
    "- 使用　(?:.|\\n)　指定一个非捕获组\n",
    "- 使用　re.DOTALL　指定.包括\\n符　也可"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[' this is a\\nmultiline comment ']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text2 = '''/* this is a\n",
    "multiline comment */\n",
    "'''\n",
    "comment = re.compile(r'/\\*(.*?)\\*/', re.DOTALL)\n",
    "comment.findall(text2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.9 将Unicode文本标准化\n",
    "### 2.10 在正则式中使用Unicode\n",
    "### 2.11 删除字符串中不需要的字符\n",
    "- strip() 方法能用于删除开始或结尾的字符\n",
    "    - lstrip; rstrip\n",
    "- replace()\n",
    "- re.sub()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hello world'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = ' hello world \\n'\n",
    "s.strip()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.12 审查清理文本字符串\n",
    "- str.translate()\n",
    "- 重新映射或者删除\n",
    "- 复杂的字符，这个更快"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'pýtĥöñ\\x0cis\\tawesome\\r\\n'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = 'pýtĥöñ\\fis\\tawesome\\r\\n'\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'pýtĥöñ is awesome'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "remap = {\n",
    "    ord('\\t') : ' ',\n",
    "    ord('\\f') : ' ',\n",
    "    ord('\\r') : None,  # Deleted\n",
    "    ord('\\n') : None # Deleted\n",
    " }\n",
    "\n",
    "s.translate(remap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.13 字符串对齐\n",
    "- ljust; rjust; center\n",
    "- format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('hello world---------', '+++++++++hello world')"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = \"hello world\"\n",
    "text.ljust(20, '-'), format(text, '+>20')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.14 合并拼接字符串\n",
    "- join()\n",
    "- +, format()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Is Chicago Not Chicago?'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def sample():\n",
    "    yield 'Is'\n",
    "    yield 'Chicago'\n",
    "    yield 'Not'\n",
    "    yield 'Chicago?'\n",
    "' '.join(sample())    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.15 字符串中插入变量\n",
    "- format(); format_map()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hang has 2 messages.'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'{name} has {n} messages.'.format(name='hang', n=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.16 以指定列宽格式化字符串\n",
    "- textwrap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Look into my eyes, look into my eyes, the eyes, the eyes, the eyes, not around the eyes, don't look around the eyes, look into my eyes, you're under.\""
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = \"Look into my eyes, look into my eyes, the eyes, the eyes, \\\n",
    "the eyes, not around the eyes, don't look around the eyes, \\\n",
    "look into my eyes, you're under.\"\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(\"Look into\\nmy eyes,\\nlook into\\nmy eyes,\\nthe eyes,\\nthe eyes,\\nthe eyes,\\nnot around\\nthe eyes,\\ndon't look\\naround the\\neyes, look\\ninto my\\neyes,\\nyou're\\nunder.\",\n",
       " 71)"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import textwrap\n",
    "import os\n",
    "l = os.get_terminal_size().columns\n",
    "textwrap.fill(s, 10), l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.17 在字符串中处理html和xml\n",
    "- 处理 <, >,  &, &entity\n",
    "- html.escape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('<html><body><title>hello</title></body></html>',\n",
       " '&lt;html&gt;&lt;body&gt;&lt;title&gt;hello&lt;/title&gt;&lt;/body&gt;&lt;/html&gt;')"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import html\n",
    "s = '<html><body><title>hello</title></body></html>'\n",
    "s, html.escape(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Spicy \"Jalapeño\".'"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = 'Spicy &quot;Jalape&#241;o&quot.'\n",
    "\n",
    "from html import unescape\n",
    "unescape(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.18 字符串令牌解析\n",
    "- text = 'foo = 25 + 42'\n",
    "- 对应：tokens = [('NAME', 'foo'), ('EQ','='), ('PLUS', '+'), ('NUM','42')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "re.compile(r'[a-zA-Z]*|\\d+|\\+|=', re.UNICODE)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import re\n",
    "# 使用正则匹配令牌\n",
    "NAME = r'[a-zA-Z]*'\n",
    "NUM = r'\\d+'\n",
    "PLUS = r'\\+'\n",
    "EQ = r'='\n",
    "\n",
    "master_pat = re.compile('|'.join([NAME,NUM,PLUS,EQ]))\n",
    "master_pat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# scanner 该方法会不断调用match()进行扫描\n",
    "sca_obj = master_pat.scanner('foo=42')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'re.Pattern' object has no attribute 'lastgroup'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-2e6e343a3b2c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msca_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlastgroup\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgroup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m: 're.Pattern' object has no attribute 'lastgroup'"
     ]
    }
   ],
   "source": [
    "sca_obj.match(), _.lastgroup, _.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.19 实现一个简单的递归下降分析\n",
    "### 2.20 字节字符串上的字符串操作\n",
    "- 大多方法与字符串相同\n",
    "- 不同：索引取值，输出格式，格式化操作，\n",
    "- 注意：要编码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(72, b'Hello World', 'Hello World')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s  = b'Hello World'\n",
    "s[0], s, s.decode('ascii')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
